Imaging sensor

ABSTRACT

The invention relates to imaging 3-D time-of-flight sensors that are particularly suitable for the detection of objects and individuals in three-dimensional space. This capability results from the basic principle. In each image point of the sensor, the distance to the object located in the observation space is determined by using the propagation time (time of flight) of light pulses. The sensor therefore supplies a three-dimensional image, which can be analyzed by means of suitable processing algorithms. In specific applications, in particular in the interaction of human and machine, machine and machine, or machine and space, it is necessary that the detection is carried out safely. The level of safety is classified in various standards into safety integrity levels.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a U.S. National Stage Entry of InternationalApplication PCT/EP2019/076963 filed Oct. 4, 2019, which claims priorityto German Patent Application Nos. 102018124551.3 filed Oct. 4, 2018 and102019124573.7 filed Sep. 12, 2019, which are incorporated by referencein their entireties.

BACKGROUND OF THE DISCLOSURE

3D time-of-flight image sensors (TOF sensors) are particularly suitablefor the detection of objects and people in a three-dimensional space.This ability results from the basic principle. In each image point ofthe sensor, the distance to the objects located in the detection area ofthe sensor is determined based on the transit time (also referred to astime of flight (tof)) of light pulses. The sensor thus supplies a sensoroutput signal that reflects a data set containing a distance value foreach image point (also referred to as a pixel) of the sensor thatcorresponds to the distance between the respective image point and thesection of the object surface mapped onto this image point. The sensorthus provides a three-dimensional image that can be analyzed usingappropriate processing algorithms. It is a known fact, for example, thatthe sensor output signal is analyzed in order to determine whetherobjects are located within a predetermined spatial sector. In certainapplications, one or more 3D time-of-flight image sensors can be used todetect whether a person and a machine part come dangerously close toeach other e. g. in the activity area of the machine.

In certain applications, in particular concerning the interactionbetween man and machine, machine and machine, or machine and space, itis necessary that the detection be performed extremely reliably, i. e.safely (safe).

The level of safety is classified as safety integrity levels (SIL) invarious standards:

Automobile QM ASIL-A ASIL-B/C ASIL-D — (ISO 26262) General — SIL-1 SIL-2SIL-3 SIL-4 (IEC-61508) Aviation DAL-E DAL-D DAL-C DAL-B DAL-A(DO-178/254) Railway — SIL-1 SIL-2 SIL-3 SIL-4 (CENELEC 50126/128/129)

STATE OF THE ART

3D TOF sensors are known that have at least one optical transmitter foremitting electromagnetic radiation, preferably (but not exclusively) inthe near infrared range, and an image sensor that is composed of anarrangement of radiation receivers. Each individual radiation receiverequals one pixel (image point) of the image sensor and thus of the 3DTOF sensor.

The radiation receivers are typically divided into two groups (radiationreceiver A, radiation receiver B) and arranged as a matrix.

Each radiation receiver has a photosensitive element (e. g. a photodiodeor a photo gate) and at least one memory capacity.

A typical mode of operation of a TOF sensor is as follows:

At a point in time t0, the transmitter is activated and then emits alight pulse for a short period of time ti (e. g. 30 ns) with a lightpulse duration corresponding to ti. At the same time, the radiationreceiver A is also activated for a short activation period tA (e. g.also 30 ns). The light pulse is reflected on an object, and theradiation receiver A registers it offset by the time of flight tof.During the activation period tA, the energy of the reflected light pulsecauses charges—namely photoelectrons—to be released in thephotosensitive element. The charge as a result of registration of thelight pulse (corresponding to the number of released photoelectrons) isstored on the storage capacity (A1). The receiver A can only receivethat part of the reflected light pulse which corresponds to the timedifference tA−tof, since the impingement of the reflected light pulse onthe photosensitive element is delayed by the time of flight tof ascompared to the start of the activation period, and the activationperiod and the pulse duration of the reflected light pulse onlypartially overlap temporally for an overlap period. The duration of theoverlap is inversely proportional to the transit time or time of flightof the light pulse (see FIG. 1). Correspondingly, the number of releasedphotoelectrons is at least approximately inversely proportional to thetransit time or time of flight of the light pulse.

The radiation receiver B is activated with a time delay A=t0+tA (again30 ns in the example), i. e. directly after completion of the firstactivation period tA. The radiation receiver B is active for a secondactivation period tB (also 30 ns in the example). Again, the receiveronly registers part of the reflected light pulse, which corresponds tothe time difference Δ=tof+ti−tA (if ti=tA, this is tof) and is thusproportional to this time difference (the transit time or time of flightof the light pulse). The corresponding charge quantity is stored on thestorage capacity B1.

Since the reflected signals can be very weak depending on thereflectance of the reflecting object and the distance, it is necessaryto provide an amplification mechanism. This is achieved by repeating themeasurement described above many times, thereby accumulating therespectively registered charge quantity on the storage capacity until asufficient signal level is reached.

The distance can be calculated as follows:

E1=P(R)*kref*(T−tof)

E2=P(R)*kref*(tof+T−T)=P(R)*kref*tof

where:

-   -   E1; E2—energies per scanning for receiver A or B    -   T=tA=tB=ti−max. time of flight=30 ns for a maximum distance of        4.5 m (uniqueness range)    -   kref—reflection coefficient of the target    -   tof—time of flight of the pulse    -   P(R)—reflected pulse power at kref=1    -   P(R)—reflected pulse power at kref=1.

The charge quantities are proportional to the respective energy valuesper scanning process.

In simplified terms, the following could be written for 200accumulations:

With

     A = ?pE₁      B = ? pE₂?indicates text missing or illegible when filed

-   -   A; B—charge quantities accumulated in storage capacity A resp. B    -   p—is the proportionality factor; its specific properties are not        relevant for these considerations.

By way of scaling (division by the sum of the charge quantitiesaccumulated in the two capacities), normalized charge quantities Q1 andQ2 that are not dependent on the reflectance can be formed.

Q1=A/(A+B) (elimination of the dependence on the degree of reflectancethrough formation of quotients)

Q2=B/(A+B)

a value for the time of flight can be generated from the normalizedcharge quantity for each capacity:

tof1=(1−Q1)×T

tof2=Q2×T

In the ideal error-free case, the two times of flight tof1 and tof2determined in this way are identical: tof 1=tof 2. Possible measurementerrors can be reduced by way of averaging:

tof=½(tof 1+tof2) (average)

The measured distance between the sensor and the reflecting objectresults in:

S=½(c*tof)c−speed of light

An extended variant of the described structure is one in which eachreceiver has two storage capacities (receiver A via Aa and Ab; receiverB via Ba and Bb). This property results in additional functions.

First Option:

The receivers A and B are operated identically. The storage capacitiesAa and Ab are switched in an offset manner—Ab takes over the function ofB (as described above).

Instead of two photo receivers, only one photo receiver with two storagecapacities is used:

     Aa = ? pE₁      Ab = ? pE₂?indicates text missing or illegible when filed

If a suitable structure is used, the advantage of this type ofarrangement can be a higher density of the receivers (betterresolution).

Second Option:

The additional storage capacities are used for 4-phase scanning of thesignal. While FIG. 1 describes signal scanning with two integrationperiods (2 phases), the signal can now be scanned four times, with thescanning points being offset by ¼ of the signal period; see FIG. 2.Periodic pulse sequences with a cycle duration T can also be used as thetransmitted signal for the embodiment version.

The phase shift φ of the reflected signal caused by the time of flighttof can be calculated as follows using the charge quantities Aa, Ab, Baand Bb accumulated in the capacities:

$\varphi = {\tan^{- 1}\frac{{Aa} - {Ab}}{{Ba} - {Bb}}}$

Strictly speaking, calculation of the phase angle only applies tosinusoidal signals.

The phase shift φ can then be used to calculate the distance S betweenthe sensor and the reflecting object:

$S = \frac{\varphi*T}{2\pi}$

T—is the period of the periodic signal in this case.

The relationships shown for the second option also apply if a sinusoidalform is selected as the periodic signal.

The specific application and the system design of the TOF sensor willlargely determine which method is appropriate.

While the method described in FIG. 1 is generally used with verypowerful pulses and a very small duty factor (e. g. >25 W pulse power;1:1000), the pulse sequences shown in FIG. 2 or those with a duty factorof 1:1 are used for outputs of 1 W to 4 W of pulse power. A similarprocedure is used when choosing the peak power for the sinusoidalmodulation of the light source. It is necessary to observe thelimitation of the applied power by adhering to the requirements for eyesafety.

Use of Active Pixel Sensors (APS)

What is known is that an active pixel sensor (APS) needs to be providedfor each pixel. An active pixel sensor typically has a photodiode andseveral (field effect) transistors. Light impinging on the photodiodereleases charges in the barrier layer of the photodiode so that the(reverse) voltage across the diode decreases as a result of the incidentphotons. An active pixel sensor is operated as follows: At the beginningof the activation period, the voltage across the photodiode is set to adefined initial value by means of the reset transistor. During theactivation period, the junction capacitance of the photodiode isdischarged by the photocurrent caused by the incident reflected lightpulse. The voltage across the photodiode drops proportionally to theintensity of the reflected light pulse and to the duration of that partof the light pulse which falls within the activation period of theactive pixel sensor. At the end of the activation period, the voltagevalue of the voltage drop across the photodiode is read out andtransmitted to analog post-processing or immediately to ananalog-to-digital converter (ADC). For this purpose, each image elementhas an amplifier transistor which, by means of the selection transistor,is usually switched in columns to a read-out line common to all imageelements in a row. Active pixel sensors can be implemented in CMOS(Complementary Metal Oxide Semiconductor) technology. Since charges inthe barrier layer of the photodiode can also be released by thermalprocesses, for example, there is typically a signal noise thatsuperposes the actual photo signal.

Correlated Dual Sampling (CDS)

To reduce the signal noise mentioned, it is known to measure the voltageacross the photodiode not only once at the end of the activation period,but then again following the resetting of the photodiode to restore thefull reverse voltage (dark voltage) in order to obtain a referencesignal that can compensate the noise signal components at leastpartially.

Sensors that implement such correlated dual sampling have acorresponding circuit for each pixel, which is referred to as the CDSstep here. The CDS step can be implemented by means of operationalamplifiers, for example.

Components of a Pixel

A pixel of an image sensor can have the following components:

-   -   an active pixel sensor (also referred to as an APS cell here),    -   a CDS step that is assigned to the APS cell, and    -   a sample-and-hold stage (S&H) for reading the pixel.

SUMMARY OF THE DISCLOSURE

The object of the present invention is to design the structure of a TOFsensor described above in such a way that a dangerous failure is veryunlikely and can preferably only occur with a probability of less than 1in 1100 years (SIL3 or Cat 4, PL e pursuant to ISO 13849-1).

To achieve this object, the application proposes a 3D TOF with at leastone pixel matrix that has a pixel structure divided into at least twosignal paths (channels A and B), wherein the signal paths are routed insuch a way that the signal transit times are identical for both channelsin accordance with the pixel location (H tree), and wherein each signalpath has its own independent analog signal output (signal A and signalB), and wherein the 3D TOF sensor additionally has at least two opticaltransmitter groups, with each transmitter group comprising at least onetransmitter.

The transmitters are preferably designed to emit in infrared light sothat they operate in the infrared spectral range of light.

A separate logic is preferably provided for each signal path of thepixel matrix (logic of the signal path A and the signal path B) for thecontrol. The logic of each signal path is preferably designed todetermine amplitudes and distance values through processing of thesignal values of both channels by exchanging the channel data crosswise.

According to a first variant of the application, the optically sensitivesurfaces of the pixels of the first channel (pixel A) and the pixels ofthe second channel (pixel B) are optically connected in such a way thatboth pixels always absorb the same amount of light.

The optically sensitive surfaces of the pixels of the first channel(pixels A) and the pixels of the second channel (pixels B) can beconnected by means of a diffuser that distributes the amount of lightevenly across the optical surfaces of one pixel each from channel A(pixels A) and of one pixel from channel B (pixels B). This method isreferred to as optical binning here. In this case, the signal paths ofthe pixel groups each comprising one pixel A and one pixel B are notelectrically connected. In an application, the pixel matrix can be readout by addressing the pixels in channel A and channel B in parallel.Consequently, the signal values of signal A1 and signal B1, of signal A2and signal B2; . . . ; of signal An and signal Bn are always at theoutput of the pixel matrix at the same time.

A comparator stage is preferably provided, which is designed to comparethe signal values that are present in parallel at the outputs of thepixel matrix. In doing so, the digitized output value of the comparatoris preferably compared with a default value in at least one of thelogics A or B. If the two signal values show a big difference and thecomparator reference value for the difference is thus exceeded, forexample, the logic preferably generates a fail signal and the read-outprocess of the pixels of the 3D TOF sensor is aborted.

An analog-to-digital converter (ADC) is preferably provided that isassigned to channel A or channel B and designed to digitize the analogsignal values A and B in the respective channel and transmit them to thelogic of channel A or the logic of channel B. The digitized channelvalues can then be compared. The channel data are preferably exchangedcrosswise for this purpose. If the calculated difference value exceedsthe reference value, the logic preferably generates a fail signal andthe read-out process is aborted.

The 3D-TOF sensor is preferably designed to perform an additionalspecial signal processing step in such a way that the functionality ofthe photo receivers is checked by comparing the signal values A and thesignal values B in the additional special signal processing step. Whenthe transmitters (of at least one transmitter group) start, the pixels Aand the pixels B are simultaneously activated or, more precisely, thestorage capacities of the respective pixels are activated. For thetwo-phase mode, the activation period corresponds to twice the length ofthe light pulse duration, and in the four-phase mode it correspondsexactly to the length of the light pulse duration. As a result of thissignal step, at the output of the pixel matrix with optically binnedpixels, the same signal values are expected at the output of the twochannels A and B. Identical signal values (the comparator threshold isnot reached) are an indication of the functionality of the pixels.

The 3D-TOF sensor is preferably designed to perform a further specialsignal processing step in which the functionality of the transmittergroups is checked. When the transmitters (only one group oftransmitters) are started, either pixels A or pixels B are activated or,more specifically, the corresponding storage capacities of therespective pixels are activated. The activation period is performed withtwice the activation period as compared to the image cycle. For thetwo-phase mode, the duration of the activation period corresponds totwice the light pulse duration, and in the four-phase mode itcorresponds exactly to the light pulse duration. As a result of thisadditional signal step, at the output of the pixel matrix with opticallybinned pixels, different signal values are expected at the output of thetwo channels A and B.

The difference in the signal values at the output of the analog pixelmatrix is an indication that the respective transmitter group has sent alight pulse and is functional.

Once verification has been provided for one transmitter group, the sameprocedure can be used for the other transmitter group. Alternating thetransmitter groups between two image cycles is a preferred method.

The 3D TOF sensor is preferably designed in such a way that the start ofan image cycle occurs with the test of the first transmitter group(claim 10), and that the test of the photo receivers (claim 9) thentakes place, followed by the actual image process. When the picturecycle is completed, the next picture cycle starts with the next (i. e.at least the second) transmitter group. This change of transmittergroups can be continued as needed.

According to one embodiment variant, the 3D TOF sensor can be designedto save the signal step of testing the first transmitter group (claim10) by the transmitter groups operating in at least two differentwavelength ranges (which may overlap) and the pixel matrix being set upin such a way that pixel A and pixel B are fixed to one of the twodifferent wavelength ranges by a corresponding optical filter, with therespective optical filter only allowing one of the two wavelength rangesto pass through. A particularly preferred solution is the use of the 850nm wavelength for the first transmitter group and the pixels of channelA and 940 nm for the second transmitter group and the pixels of channelB. If the signal step according to claim 7 is now performed, the firstand second transmitter groups are activated simultaneously in this case.The pixels of channel A receive the reflected light pulse in the firstwavelength range (850 nm), and the pixels of channel B receive thereflected light pulse in the second wavelength range (940 nm). If theoutput of channel A and channel B then has signal values of the samesize or signal values at a predetermined ratio for the respectiveadjacent pixels addressed, it is a confirmation that both the addressedphoto receivers of the two channels and the two transmitter groups arefunctional.

Preferably, in this case, an additional optical binning can also beprovided after the two wavelengths to improve the correspondence of thesignal values of the optical filtering described above.

The 3D TOF sensor can preferably be improved by adapting the area of thephotoreceptors of channel A and channel B according to the selectedwavelength in such a way that the different quantum efficiency fordifferent wavelengths is compensated, i. e. that signal values of thesame size are present at the output of channels A and B with an intact3D TOF sensor.

Each channel preferably has its own processor that is connected to thesignal path of the logic of the channel. Furthermore, the processors arepreferably connected to the input of the respective other channel. Thecalculated amplitudes and the distance values that the respective logichas determined are thus transmitted to each processor.

The logical operations of each channel logic are preferably tested insuch a way that each processor calculates at least one test value(random number) completely independently of the other processor.Typically, one test value is calculated for the two-phase mode and twotest values for the four-phase mode. The processor transmits each testvalue to the logic of the same channel. Using the transmitted testvalues and the test values of the other channel, the logic calculates atleast one output value, preferably two. The same operations are used forcalculating the output value and for calculating the amplitudes anddistance values. The output values are transmitted to the processor ofthe respective channel. The processor in turn calculates the expectedoutput values and checks the output values received from the logic ofits channel against the output values it has calculated and issues afail signal in the event of a deviation.

In a particularly preferred embodiment, one of the two processors ispreferably a safety-related processor. A safety-related processor has atleast two independent reactionless computing cores and processes twoinputs on one output (1oo2—one out of two). This architecture ispreferably used for dividing the image operations into a safe and anon-safe image processing process. The safe process only examineswhether deviations of individual pixels from the expected standardposition occur in the defined field of view of the 3D TOF sensor(deformation of the 3D point cloud). If such deviation occurs, an alarmsignal is emitted.

Complex image processing takes place on the non-safe processor with thegoal of identifying the actual cause of the deformation of the pointcloud (e. g. recognizing the movement of a person, a hand, a head,etc.).

A second variant of the application proposes a 3D TOF sensor that ispreferably characterized by the following features:

The 3D TOF sensor has an image sensor with a pixel matrix, wherein

-   -   the pixel matrix has at least two separate independent blocks        and transmits the data in at least two separate signal paths;        and/or    -   the pixels of the pixel matrix of the image sensor have at least        one photosensitive receiver and one storage cell for storing        charge carriers; and/or    -   the signal paths are at least made up of an analog-to-digital        converter (ADC), a programmable logic, storage cells and a        processor connected thereto; and/or    -   the image sensor has at least one global shutter for all blocks        that can be released by a control signal of the programmable        logic and can set a minimum shutter time of less than 100 ns;        and/or    -   the signal paths are wired independently and reactionlessly of        each other; and/or    -   the signal paths are wired such that the data of the logics or        the processors can be exchanged crosswise; and/or    -   each block of the image sensor is assigned its own imaging        optics such that the same image of the object side is mapped on        the blocks. The distance between the optics is selected to be so        small that the parallax error cannot be resolved; and/or    -   each block is assigned at least one light source that is        synchronized with the global shutter via the logic.

The image sensor is preferably characterized in that each block has aseparate global shutter that can be controlled by the logic of theassigned signal paths.

The image sensor is preferably characterized in that the global shuttersof the blocks only depend on a single logic signal of a single signalpath and their dependency is set by adjustable delay times (registervalues).

Alternatively, or additionally, the image sensor is further preferablycharacterized in that, for each block,

-   -   a narrow optical spectral range is assigned by an optical filter        or a system of optical filters; and/or    -   the system of optical filters is formed by filters directly on        the image sensor and at least one further filter in the imaging        system of the optics; and/or    -   the optical filters on the blocks of the image sensor ensure, as        edge filters, a sharp spectral separation of the blocks, while        the filter in the optical imaging system is designed as a band        filter and includes the spectral ranges of the blocks; and/or    -   each spectral range of the blocks is assigned a light source        which emits light precisely in the wavelength that corresponds        to the transmission range of the optical filters of the        corresponding block.

This corresponds to a method of reliably recording 2D image data with animage sensor of the aforementioned type, comprising the following methodsteps:

-   -   In a first processing step, the global shutter is released by a        previously defined exposure time. As a result, there is a        charged image in the memory of the pixels on both blocks that        corresponds to the object image.    -   In a second processing step, the image data of the first block        is transferred to the processing line of the first block        (readout block 1) and the image data is transferred to the        processing line of the second block (readout block 2), etc. As a        result of this step, the image data is now available in digital        form in the blocks.

The invention will now be explained in more detail using exemplaryembodiments and referencing the figures.

BRIEF DESCRIPTION OF THE FIGURES

The figures show the following:

FIG. 1: illustrates the distance measurement according to the TOFprinciple, which is based on a time-of-flight measurement for an emittedlight pulse (emitted pulse);

FIG. 2: illustrates a transit time measurement similar to that in FIG. 1with four-phase scanning of the reflected light pulse;

FIG. 3: illustrates the basic structure of a TOF sensor according to theapplication with two parallel paths and a cross comparison of these;

FIG. 4: illustrates how the basic structure shown in FIG. 3 can beimplemented on the pixel level of a 3D TOF sensor;

FIGS. 5a-5c : illustrate different variants of how the pixels of a pixelmatrix can be divided into two channels (channel A and channel B);

FIGS. 6a, 6b : illustrate in a sectional view how optical binning can beimplemented;

FIG. 7: uses the example of the pixel matrix from FIG. 5a to illustratehow the pixels of channels A and B to be evaluated in pairs can beoptically connected in parallel by way of optical binning;

FIG. 8: illustrates the simultaneous scanning of the reflected lightpulse;

FIG. 9: shows a circuit diagram for scanning a pixel matrix according tothe FIGS. 6 and 7 based on the principle depicted in FIG. 8;

FIG. 10: shows that the light for emitting the light pulses is alsoduplicated and can have two transmitter groups with two transmitterseach to be used alternately;

FIG. 11: illustrates the scanning of only one of the pixels, i. e. pixelA or pixel B, instead of the simultaneous scanning of pixels A and Bshown in FIG. 8 so as to obtain a measure for the amount of light of theactivated transmitter group;

FIGS. 12a, 12b : illustrate that the respective pixels A and B of thepixel matrix can be sensitive to different wavelengths or differentwavelength ranges;

FIG. 13: shows that the surface area of the pixels in a pixel matrixaccording to FIG. 12a or 12 b can have a different size, if the quantumefficiency of the pixels is different for the respective wavelength, inorder to obtain the depicted output signals with comparable levels bycompensating a lower quantum efficiency with a larger pixel area;

FIG. 14: shows a logic circuit with a further comparator for testingchannels A and B, which respectively operate in two wavelength ranges;

FIG. 15: shows that, instead of two identical processors A and B forevaluating the signals of channels A and B, two different processors canbe used, one of which is a safety-related processor;

FIG. 16: shows the basic structure of an image sensor in which theoperational amplifier of the CDS step, by expanding the circuit, alsoacts as a comparator for comparing the signal values of pixels A and Bon the analog pixel level;

FIG. 17: illustrates the read-out cycle;

FIG. 18: shows the CDS step of the image sensor from FIG. 16 in a firstswitching status;

FIG. 19: shows the CDS step of the image sensor from FIG. 16 in a secondswitching status;

FIG. 20: shows the CDS step of the image sensor from FIG. 16 in a thirdswitching status;

FIG. 21: shows a high-resolution image sensor, which has severalparallel read-out channels that are used for reading out the pixelsblock by block;

FIG. 22: shows an image sensor with two pixel blocks A and B, with eachone being sensitive to different wavelength ranges;

FIG. 23: illustrates that testing of the function of the pixel matrixcan be carried out on the block level in exactly the same way as on thepixel level (cf. FIG. 8);

FIG. 24: shows an image processing circuit for reading out a pixelmatrix that is divided into two blocks A and B;

FIG. 25: shows an image sensor similar to that in FIG. 22, for which anadditional optical channel is provided; and

FIG. 26: illustrates the serial scanning of the reflected light pulse bymeans of an image sensor that has only a single global shutter.

DETAILED DESCRIPTION OF THE FIGURES

The starting point of the invention is a general arrangement asdescribed in EN ISO 13849 (machine safety), according to which SIL-3(Cat. 4; PL e) can be achieved by designing the system with at least twosignal paths so that the integrity can be achieved by a cross comparisonof the two parallel signal paths; see FIG. 3.

According to the basic structure of a 3D TOF sensor 10 shown in FIG. 3,it has two input units 12A and 12B, which can be created by pixels; cf.FIG. 4. The output signals generated by the input units, for example theoutput signals of the pixels, are respectively transmitted to a logic14A and 14B that evaluate the output signals of the input unit 12A and12B. The logics 14A and 14B are interconnected for cross-comparisonpurposes. In the context of the cross comparison, it is in particulartested whether the logics 14A and 14B each deliver the same outputsignal. If the result of the cross comparison is positive, the signalsreceived by the input units 12A and 12B and processed by the logics 14Aand 14B are output to a respective output unit 16A and 16B.

The 3D TOF sensor as per FIGS. 3 and 4 thus has two parallel signalpaths, one of which is created by the input unit 12A, the processinglogic 14A and the output unit 16A, and the second signal path is createdby the input unit 12B, the processing logic 14B and the output unit 16B.The first signal path thus creates a channel A, and the second signalpath creates a channel B. The proposal now is to apply this parallelsystem to a 3D TOF sensor.

In the following, the terms “signal path” and “channel” are thereforeused synonymously.

First Variant

In the context of the first variant, the principle with at least twopixels (pixels A and B) described above is applied in such a way thatcontinuous parallel processing with comparison of the signal paths A andB is achieved. Comparison of the signals is performed at differentlevels of the signal processing; see FIG. 4.

All pixels A and B are arranged as a matrix in respectively the samerows, columns, or as a chessboard.

The signal paths are strictly separated according to channel A andchannel B. This applies to both the analog and the digital signal path,regardless of whether the receiver matrix integrated in silicon isexclusively analog, or analog and digital; see FIG. 5.

There are thus both implementation options, namely integration of onlythe analog circuit parts in silicon and use of an external logic on anFPGA, or full integration of all components in silicon or anothersemiconductor material.

Optical Binning/Testing of the Receiver Function

The combination of several pixels on the circuit level is a knowntechnical fact. This procedure is used to:

-   -   decrease the resolution,    -   increase the image speed, or    -   improve sensitivity and reduce noise.

Optical binning is introduced as a new feature to ensure that pixels Aand B receive identical optical power (image).

Optical binning can be achieved by placing a homogenizing diffuser overpixel A and pixel B. This arrangement can be accomplished in such a waythat only the optically sensitive surfaces (photo receivers) involved inpixel A and B are optically connected. However, it is also possible toplace the diffuser over the entire pixel geometry. Another variantresults from the use of micro-optics as they are already frequently usedto increase the fill factor in CMOS image sensors. The micro-optics arethen placed over the diffuser.

In a further embodiment variant of the first variant, the diffuser andmicro-optics can also be implemented in one element on a micro-opticalbasis.

Based on the matrix arrangement of pixels A and B, A1 and B1, A2 and B2. . . An and Bn are optically connected to each other, provided theanalog signal path of A1 and B1 or A2 and B2, An and Bn are designed tobe identical in length to maintain the same signal propagation times.This requirement also applies to the digital triggering signals of theswitching transistors of the respective optically binned pixels; seeFIG. 6 and FIG. 7.

The functionality of the parallelized photo receivers is tested with anadditional signal step. The object of this signal step is to testwhether channels A and B deliver an identical signal value. These signalvalues must be well above the noise level (zero signal).

This is achieved by simultaneously scanning the reflected light pulse.For the two-phase mode, the scan time corresponds to twice the length(see FIG. 8) of the light pulse duration, and in the four-phase mode itcorresponds exactly to the length of the light pulse duration. The cycleof sending and scanning the reflected light pulse should preferably berepeated several times in this case as well, until a sufficient chargequantity has accumulated in the memories of both pixel A and pixel B.

The circuit diagram is outlined in FIG. 9. The memories of the opticallybinned pixels (A1 and B2) are selected simultaneously after completionof the integration phase and applied to the output of the analog matrix.The charge quantities of pixels A1 and B1 are compared in the analogsignal path for the first time (e. g. by using a comparator) to checkwhether the differential voltage between A and B is below a previouslydefined threshold value. Depending on the number of memories in thepixel (two-phase or four-phase), this cycle can be performed once ortwice per pixel. It is also possible to implement a circuitryparallelization of the memories a and b.

The signal values are then digitized in an analog-to-digital converter(ADC A and ADC B; see FIG. 9) and compared a second time. The operationscan be performed both by the downstream logic and/or by the processor Aor B downstream from the respective logic A or B. The logic A and thelogic B differ from the processor A and the processor B in their form ofimplementation. The logics are circuits that implement the logicaloperations on the circuit level and can, for example, exist in the formof an FPGA (Field Programmable Gate Array). The mode of operation of theprocessors, on the other hand, is determined by sequence programming (e.g. in the form of firmware). The processors can be digital signalprocessors (DSP), for example.

If the defined differential value is exceeded in the analog or digitalrange, a fail signal is generated in the logic block and the imagerecording is stopped. This fail signal is transmitted directly to theprocessor interfaces (output) via the signal path.

The processor output in connection with a safe control can then lead amachine to a safe stop, see FIG. 9.

Duplicated implementation of the light sources/Testing of thefunctionality of the transmitters

Just like the signal path is duplicated, the lighting of a 3D TOF sensorwith optically binned pixels is also duplicated. At least twotransmitter groups are used alternately; see FIG. 10.

A transmitter group consists of at least one transmitter. FIG. 10 showsthe duplicated implementation of the transmitter group, in each caserepresented by two transmitters. The transmitters 1 and 3 as well as 2and 4 are operated in parallel and alternately. LEDs or laser diodes canbe used as transmitters (e. g. VCSEL); see FIG. 10.

Extended Cycle for Testing the Transmitter Function

The optical binning and additional signal step can be used to testwhether the photo receiver and the signal processing pipeline aredelivering the same signal values. The parity of the signal values canbe determined by way of a difference value, which should be below athreshold value to be specified.

In this step, no determination can be made about the functionality ofthe transmitters. Another additional processing step can provideinformation about this.

If, instead of the simultaneous scanning of the reflected light pulsefrom pixel A and pixel B described in FIG. 8, only one pixel, i. e.either A or B, is scanned, a signal above a threshold value is generatedwhen the difference value is determined in the comparator stage (analog)or in the digital logic part; this signal provides a measure of theamount of light from the activated transmitter group; see FIG. 11.

In the running image cycle, the signal steps according to FIG. 8 andFIG. 11 are put before the actual image generation so as to monitor thesignal paths.

Testing transmitter Testing Image Testing transmitter group 1 receiverprocess group 2

The transmitter group 1 and the transmitter group 2 are activatedalternately during the cycle.

Alternative Test Mechanism of the Transmitter Function by Splitting theTransmitter Wavelengths

A particularly advantageous embodiment of the continuous parallelism ofthe signal routing can be achieved by the transmitter groupstransmitting in different wavelengths, while the pixels A and B canreact selectively to the respectively different wavelengths; see FIGS.12a and 12 b.

The advantage of this method is that the signal step described withreference to FIG. 11 does not have to take place iteratively once forthe first and for the second transmitter group, but that this step cantake place simultaneously since both transmitter groups can be activatedat the same time and recognized selectively because of the waveselectivity of the receivers.

The optical binning then occurs in a wavelength-selective manner. Thismeans that a typical implementation could use a narrow band filter andhomogenization function with a central wavelength of 850 nm, and for thesecond receiver group with a central wavelength of 940 nm.

In a special embodiment, the optically sensitive surfaces of the photoreceivers could be designed in such a way that the different quantumefficiencies of the different wavelengths are balanced (in silicon, thequantum efficiency for 940 nm drops by about 30% compared to 850 nm). Alarger sensitive area of the receiver with low quantum sensitivity couldoffset the disadvantage; see FIG. 13.

FIG. 12 shows different arrangements of the wave-selective solution. InFIG. 12a , A and B operate in both wavelengths. The functionality of Aand B is checked for each wavelength. As a consequence, anothercomparator can be used. Depending on which wavelength is used, eitherlogic A or logic B processes the comparator signal (FIG. 14).

If the filter arrangement according to FIG. 12b is used and thetransmitter groups with the two different wavelengths are activatedsimultaneously in the test step, the transmitter test and the receivertest can be performed at the same time. Since the difference signal atthe comparator output between channels A and B is smaller than a definedthreshold value, it can be concluded that both transmitter groups andthe photo receivers of both channels are functional.

Function of the Logics A and B

Logic A supplies the control signals for the photo receivers of channelA, for example to control the switches of the CDS step for signal path Ashown in FIGS. 18 to 20. Logic B supplies the control signals for thephoto receivers of channel B, for example to control the switches of theCDS step for the signal path B shown in FIGS. 18 to 20. Asynchronization of the logics ensures that the correct pixels pairs areaddressed in the read-out process. The validity of the signal values isdetermined by means of the comparator signal.

In a further processing step, the digital difference signal of thechannels is calculated separately in each logic. To this end, the logicsexchange the digitized channel data, provided they are considered to bevalid. If the digitized difference signal of pixels A and B exceeds thethreshold value, a fail signal is transmitted to the correspondingprocessor.

Both logics calculate the signal amplitude values and the distancevalues independently. The calculated values of logic A are transmittedto processor A, and the values of logic B to processor B. In addition,the calculated values are delivered crosswise. Each processor thencompares the amplitude and distance values. If the comparison results ina deviation, the respective processor sends a fail signal.

Function Test of the Logics

Between the image cycles, the processors send a digital input value(random number) to the upstream logics. The logics introduce this randomnumber into their processing pipeline and use it to calculate an outputvalue that is checked for correctness by a processor routine for eachprocessor. Processing includes the exchange of input values between thelogics. The test process for each logic thus uses the input values ofchannel A and channel B for the calculation.

The calculating functions correspond to the operations for determiningthe distance and amplitude values. A fail signal is output via therespective processor if the comparison in a processor results in anerror.

Function of the Processors A and B

In a first embodiment, both processors perform identical operations.Each processor has its own memory and own interfaces.

The outputs OUT A and OUT B are processed on a downstream level (e. g.safety-related control).

In a second embodiment, which is particularly advantageous for compleximage processing operations,

e. g. motion analyses of people, a safety-related processor is used forone of the processors (e. g. channel A).

A safety-related processor has itself an internal parallelizedreactionless architecture. These types of processors are known (e. g.HICore 1—https://www.hima.com/en/products-services/hicore-1/); they havea so-called 1oo2 architecture (one out of two), for example. In thiscase, the processing is divided into a safe process and a non-safeprocess.

The division will be explained using the following example.

When monitoring a work area, it is important to prevent a hand fromentering a safety zone or to stop a machine if a hand enters the zone.

Since recognizing a hand requires certain complex image processingoperations (computing time), the process is divided in such a way that acertain area is defined as a three-dimensional safety zone. Thesafety-related processor consequently only monitors the position of thepoint cloud in the safety zone defined in the FOV (field of view). Adeviation (deformation) of the point cloud would result in a failsignal. Monitoring the position of the point cloud is a relativelysimple operation.

The other processor examines which object has entered the safety zone.The corresponding operations are complex and are implemented usingmodern high-level languages. In the case of a hand, for example, itresults in an instruction to the operator; see FIG. 15.

There is also the option, by expanding the circuit, to implement theoperational amplifier of the CDS step also as a comparator for comparingthe signal values of pixels A and B on the analog pixel level.

FIG. 16 shows the basic configuration of a corresponding image sensor.In the following FIGS. 18 to 20, the CDS steps in various switchingstates are shown in more detail.

As an example, a pixel is made up of an APS cell, a CDS and a sample andhold step here (marked with S&H in the figures).

On the one hand, the CDS step serves as an accumulator for the necessarymultiple exposures, and at the same time as a step for suppressing thenoise components from the reset of the photodiode of the APS cell aswell as for blocking background light.

The cross comparison with the adjacent pixel according to the sequencealready explained is now introduced into the CDS step.

FIG. 17 illustrates the read-out cycle.

The mode of operation can easily be explained using the function of theCDS step.

Phase 1:

In pixel A—the switches S1 a, S3 a are closed; the switches S2 a, S4 aand S5 a are open.

In pixel B—the switches S1 b, S3 b are closed; the switches S2 b, S4 band S5 b are open.

We get the following ratios:

In Pixel A:

Q′1a=C1a(Va−Vref);Q′2a=C2a(Vref−Vouta(n))

In Pixel B:

Q′1b=C1b(Vb−Vref); Q′2b=C2b(Vref−Voutb(n))

FIG. 18 shows pixels A and B in a first switching state of the CDSsteps, which they assume in phase 1.

Phase 2:

In pixel A—the switches S2 a, S4 a are closed; the switches S1 a, S3 aand S5 a are open.

In pixel B—the switches S2 b, S4 b are closed; the switches S1 b, S3 band S5 b are open.

In pixel A:

Q″1a=C1a(Vb−Vref); Q″2a=C2a(Vref−Vouta(n+1))

In pixel B:

Q″1b=C1b(Va−Vref); Q″2b=C2b(Vref−Voutb(n+1))

Using Kirchhoff's nodal rule, it can be written as follows:

In pixel A:

Q′1a+Q′2a=Q″1a+Q″2a

If the relationships for the charges are inserted into this equation,the following result is obtained after a corresponding rearrangement:

Vouta(n+1)=C1a(Vb−Va)+C2aVout(n)

-   -   Equivalently, the ratio at the output of pixel B is obtained:

Voutb(n+1)=C1b(Va−Vb)+C2bVout(n)

Phase 1 and phase two are repeated several times (n times) in order toachieve a sufficient output voltage on the capacitor C2 a or C2 b.

FIG. 19 shows pixel A and pixel B in a second switching state of the CDSsteps, which they assume in phase 2.

Phase 3—Test Phase

All switches are open, only switches Sa5 and Sb5 are closed. In thisphase, the operational amplifier acts as a comparator in both pixels.

The resulting voltage on the capacitors Ca2 and Cb2 is compared with therespective reference voltage.

FIG. 20 shows pixel A and pixel B in a third switching state of the CDSsteps, which they assume in phase 3.

The differential voltage Vb−Va accumulated multiple times is presentacross the capacitance Ca2.

The differential voltage Va−Vb accumulated multiple times is presentacross the capacitance Cb2.

We then compare 3 cases.

1. N×Vouta(n+1)=N×Voutb(n+1)—That would be the ideal case if pixel A andpixel B delivered the same value. This ideal case is only theoretical innature due to the physical properties of real components.

2. N×Vouta(n+1)≈N×Voutb(n+1)—The values at the output of pixel A andpixel B are approximately the same. One of the values is a little higheror a little lower than the other. In this case, both pixels can beconsidered to be functional.

A logical 0 appears on both output lines of the comparator.

3. N×Vouta(n+1)≠N×Voutb(n+1)—The output value of pixel A is either muchgreater or much smaller than the output value of pixel B. A pixel is tobe considered defective.

In this case, a logical 1 would be present at the pixel output A ifVb>>Va.

In the case of Vb<<Va, a logical 1 would be present at the output ofpixel B.

The CDS step is reset after the comparison, and a new cycle can start.

The switches of the respective CDS step are controlled by logic A orlogic B assigned to the corresponding signal path.

Additional Mode of Operation:

The introduced subtraction Vb−Va and Va−Vb can also be used veryadvantageously in a 4-phase operation. In this case, the subtraction canbe used to calculate the phase angle without reading out the individualvalues, as described above, by calculating the phase shift φ as follows:

$\varphi = {\tan^{- 1}\frac{{Aa} - {Ab}}{{Ba} - {Bb}}}$

The prerequisite is the use of two additional pixels so that a group offour pixels or a group of two pixels, but with a total of four memorycells, results in a complete 4-phase architecture.

Second Variant

According to a second variant, image recorders (imagers) are set up e.g. in CMOS technology with high resolution, preferably with severalparallel read-out channels; see FIG. 21. The object of this architectureis to achieve a high frame rate in spite of the high resolution. Theread-out channels usually combine either several rows or several columnsof an image sensor matrix. This type of combination will be referred toas a block or an imager block in the following;

see FIG. 22.

FIG. 22 shows the following individual components:

-   -   1—imager    -   2—optically sensitive surface    -   4—imager block A with spectral filter A    -   5—imager block A with spectral filter B    -   7—read-out channel for imager block A    -   8—read-out channel for imager block B.

The second variant takes advantage of this division of the image fieldinto read-out blocks.

An additional variant will be explained in more detail below using afirst exemplary embodiment.

The image field, which preferably has a very high resolution, e. g.1280×1024 pixels, is divided into blocks according to the read-outchannels created. Each read-out block operates with a global shutterthat can be released separately.

Each block is still assigned its own imaging optics. The distancebetween the optics is very small so that parallax is no longer resolvedeven at a small object distance on the image side. This means that thesame image of the object side is generated on each block.

This arrangement of optics and image recorder is used in a parallelizedread-out and image processing circuit according to FIG. 4.

Calculation of a Depth Image:

The exemplary embodiment is based on a two-channel set-up. The imagerecorder (imager) is divided into block A and block B. In the presentexemplary embodiment, each block has 640×1024 pixels. The digitalread-out channels for block A and block B are routed to thecorresponding logics A and B. It is irrelevant whether these logicblocks are already part of an integrated circuit or comprise an externallogic circuit, e. g. an FPGA.

The logics A and B control the respective global shutter signals andaccordingly also the lights LA and LB. The light sources are operatedsynchronously. Both sources are activated simultaneously with the globalshutter signal from block A at time t0 for the duration ti=tA. The lightpulse is reflected on a target and registered by all receivers A ofblock A offset by the time of flight tof. The charge quantity(photoelectrons) resulting from the registration of the light pulse isstored in the storage capacity (A1) belonging to each receiver. Thereceivers of block A can only pick up part of the reflected light pulsethat corresponds proportionally to the time difference tA−tof.

The global shutter of the receivers of block B is activated with a timedelay t0+tA. Receiver B is active for time tB (in the example tA=tB).The receiver also registers only part of the reflected light pulse thatcorresponds proportionally to the time difference tof+ti−tA (if ti=tA,this is tof). The corresponding charge quantities are stored in thestorage capacity B1 belonging to each receiver in block B.

Since the reflected signals can be very weak depending on thereflectance of the target and the distance, it is necessary to providean amplification mechanism. This is achieved by repeating themeasurement described above many times and accumulating the respectivelyregistered charge quantity in the associated storage capacities until asufficient signal level is reached. This process can be repeated 100times, for example.

As a result, there is an image in the storage capacities A1 of block Afor the time period t0 to tA, and there is an image in the storagecapacities B1 for the time period tA to tB.

These two images are transmitted to the respective logic via therespective read-out channel. The logics subsequently exchange the imageinformation crosswise and calculate the respective depth imageseparately, pixel by pixel, according to the above rules:

Q1=A/(A+B) (elimination of the dependence on reflectance throughformation of quotients)

Q2=B/(A+B)

toff=(1−Q1)*T

tof2=Q2*T

tof=½(tof1+tof2) (average)

S=½(c*tof)

The calculation rule is applied in such a way that pixels are alwayscalculated according to their position in the receiver matrix, i. e.A0,0 with B0,0; A0,1 with B0,1; . . . Am,n with Bm,n according to thechosen arrangement:

A 0, 0  …  A 0, n⋅  …   ⋅ Am, 0  …  Am, nB 0, 0  …  B 0, n⋅  …   ⋅ Bm, 0  …  Bm, n

At the end of the calculation, both logics each contain a distanceimage. Both distance images should be identical except for atemperature-dependent calibration factor that can be determined once.

Testing the Function by Comparing the Depth Image:

The identity of the two depth images A and B is tested in a furtheroperation by means of a pixel-by-pixel comparison. This comparison canbe performed on both logics or on the downstream processors.

A threshold value can be defined for the comparison that specifies howmany pixel pairs can deviate from a defined expected comparison value.The threshold value defines whether the pixel pair delivers the samevalues or not and what is to be regarded as equal or unequal. Inaddition, the number of unequal pairs can be used as a decisionparameter.

If the defined threshold value is exceeded on at least one processor, atleast one of the processors generates a fail signal at the output.

Comparison of the depth images A and B already adds up severalcomparison functions. These include pixel sensitivity, gate functions(integration/accumulation), read-out, ADC, and logic operations.

Validity of the full functionality of the pixel matrix can also beachieved by a pixel-by-pixel comparison of an amplitude image.

Testing the Function of the Pixel Matrix in the 2D Amplitude Image:

Just like on the pixel level (see above and FIGS. 8 and 9), testing thefunction of the pixel matrix in the 2D amplitude image can also beperformed at the block level; see FIGS. 23 and 24. This is achieved bysimultaneous scanning of the reflected light pulse in both block A andblock B. In this case as well, the cycle of sending and scanning thereflected light pulse must be repeated several times until a sufficientcharge quantity is accumulated in the memories of the pixels in Block Aand Block B.

After completion of the accumulation cycle, the images in blocks A and Bare read out and compared pixel by pixel on both logics. The comparisonvalues can be evaluated both in the logic and in the downstreamprocessors.

A threshold value can be defined for the comparison that determineswhether the pixels should be considered to be equal or unequal. Inaddition, it can be defined how many pixels pairs can deviate from eachother (not be equal) by a defined range before the fail signal isgenerated. If the defined threshold value is exceeded on at least onechannel, at least one of the processors generates a fail signal at theoutput.

Testing the Light Sources:

The described test cycle is also suitable for testing the light sources.A total of 4 light sources are used in the exemplary embodiment, ofwhich two are respectively controlled by the logic of channel A orChannel B.

The light sources can now be tested using either the sources of channelA or the sources of channel B in each test cycle.

Unlike comparing the receiver signal values by means of a crosscomparison, the light sources are compared by means of a serialcomparison in the respective channel.

In the present exemplary embodiment, the test is performed as follows:

-   1. Starting a light pulse with twice the length as compared to    recording of the depth image (e. g. 60 ns) of the light source of    channel A.-   2. Simultaneously starting the global shutters in both blocks A and    B with twice the length of the light pulse (e. g. 120 ns).-   3. Saving the images of block A (image A0) and block B (image B0) in    the logic blocks of the respective channels.-   4. Starting a light pulse with twice the length as compared to    recording of the depth image (e. g. 60 ns) of the light source of    channel B.-   5. Simultaneously starting the global shutters in both blocks A and    B with twice the length of the light pulse (e. g. 120 ns).-   6. Saving the images of block A (image A1) and block B (image B1) in    the logic blocks of the respective channels.-   7. Each pixel value A(i,j) from the sub-matrix A and each pixel    B(i,j) from the sub-matrix B during exposure to the LA source is    compared with the corresponding values from the exposure to the LB    source. A maximum difference value is defined. If this value is    exceeded, there is a defect and a fail signal can be generated. It    is also conceivable to generate the fail signal only if the    difference value is exceeded for a predetermined number of pixels    pairs. The comparison values can be evaluated both in the logic and    in the downstream processors.

The invention is explained using a second exemplary embodiment.

Just like in the first embodiment, the image field, which preferably hasa very high resolution, e. g. 1280×1024 pixels, is divided into blocksaccording to the created read-out channels. Unlike the first exemplaryembodiment, each read-out block is subdivided again, which means thateach read-out block operates with two global shutters that can bereleased separately.

Each sub-block (Aa, Ab, Ba, Bb) is also assigned its own imaging optics.The distance between the now four optics is very small so that parallaxis no longer resolved even at a small object distance on the image side.This means that the same image of the object side is generated on eachsub-block.

This arrangement of optics and imager is used in a parallelized read-outand image processing circuit according to FIG. 24.

Calculation of a depth image:

The exemplary embodiment is still based on a two-channel set-up.However, the image recorder (imager) is now divided into block Aa and Aband block Ba and Bb. In the present exemplary embodiment, each block has640×512 pixels. The digital read-out channels for block A and block Bare routed to the corresponding logics A and B.

Logics A and B control the respective global shutter signals andaccordingly also the lights LA and LB. The light sources are operatedsynchronously.

While the first exemplary embodiment describes signal scanning with twointegration periods (2 phases), the signal is now scanned four times,with the scanning points being offset by ¼ of the signal period.

With regard to the exemplary embodiment with a light pulse of 30 ns inlength, this would result in a scanning sequence that could be describedas follows: sub-block Aa from 0 to 30 ns, for sub-block Ba 30 ns to 60ns, for sub-block Ab 60 ns to 90 ns, and for sub-block Bb 90 ns to 120ns; see also FIG. 2.

The depicted relationships also apply if a sinusoidal form is selectedas the periodic signal.

Since the reflected signals can be very weak depending on thereflectance of the target and the distance, it is necessary to providethe amplification mechanism already described above. This is achieved byrepeating the measurement multiple times and accumulating therespectively registered charge quantity in the associated storagecapacities of the pixels in the sub-blocks until a sufficient signallevel is reached. This process can be repeated 100 times, for example.

As a result, there is one image for each quarter of the period in thestorage capacities of the pixels of the sub-blocks. Based on this, thephase and thus the distance S per pixel can be calculated using thefollowing rule.

φ=tan{circumflex over ( )}(−1)[(Ba−Bb)/(Aa−Ab)]

S(φ*T)(2π)

-   -   T—is the period of the periodic signal in this case.

Strictly speaking, calculation of the phase angle only applies tosinusoidal signals. Respective corrections for high linearity of thedistance characteristic need to be applied in the case of square wavesignals. One option for improving the linearity of the distancecharacteristic in 4-phase scanning can be achieved, inter alia, byscanning with T/2.

The test procedures described in the first exemplary embodiment areperformed in exactly the same way.

In the following, we will explain a third exemplary embodiment that isshown in FIG. 26.

In contrast to the previous examples, we only have one global shutteravailable for the entire pixel matrix. While two global shutters wereused for the phases of the depth image in the previous exemplaryembodiments (see FIGS. 1, 8, 11 and 23) so that two phase images can berecorded at the same time (in parallel), the exemplary embodiment shownin FIG. 26 is based on one image recorder with only a single globalshutter. In this case, the two phase images have to be recorded oneafter the other (serially), as illustrated in FIG. 26. Other than that,the same optical set-up as in the first exemplary embodiment can bearranged. The pixel matrix is divided into two blocks A and B along thetwo read-out channels. Each block is assigned its own imaging optics.The distance between the two optics is very small so that parallax is nolonger resolved even at a small object distance on the image side. Thismeans that the same image of the object side is generated on each block.

In contrast to the first exemplary embodiment, each block isadditionally provided with an optical filter or a system of filters. Thefilter or filter system limits the transmitted wavelength of each blockto a narrow spectral band that respectively corresponds to the sourcewavelength of the associated light source; see FIG. 25.

The exemplary embodiment in FIG. 25 differs from that in FIG. 22 in thatit has an additional optical channel 3 and an additional read-outchannel 6. In detail, FIG. 25 shows the following:

-   -   1—image recorder    -   2—optically sensitive surface    -   3—additional optical channel    -   4—imager block A with spectral filter A    -   5—imager block B with spectral filter B    -   6—additional read-out channel    -   7—read-out channel for imager block A    -   8—readout channel for imager block B.

For all embodiments shown, the following applies: The respective 3D TOFsensor can be designed to save the signal step of testing the firsttransmitter group in that the transmitter groups operate in at least twodifferent wavelength ranges (which may overlap) and the pixel matrix isstructured in such a way that pixel A and pixel B (or block A and blockB) are set to one of the two different wavelength ranges by acorresponding optical filter, with the respective optical filter onlyallowing one of the two wavelength ranges to pass. Using the wavelength850 nm for the first transmitter group and the pixels of channel A and940 nm for the second transmitter group and the pixels of channel Bconstitutes a particularly preferred solution.

1. A 3D TOF sensor with at least one pixel matrix that has a pixelstructure divided into at least two signal paths (channels A and B),wherein the signal paths are routed in such a way that signalpropagation times are identical for both signal paths according to apixel localization (H tree) and wherein each signal path has its ownindependent analog signal output (signal A and signal B), wherein the 3DTOF sensor also has at least two optical transmitter groups, of whicheach transmitter group contains at least one transmitter.
 2. The 3D TOFsensor according to claim 1, wherein the transmitters operate in theinfrared spectral range.
 3. The 3D TOF sensor according to claim 1,wherein a separate logic of signal path A and signal path B forcontrolling and processing the signal values originating from the pixelsof the corresponding signal path is provided for each signal path of thepixel matrix.
 4. The 3D TOF sensor according to claim 3, wherein thelogic of each signal path is designed to determine amplitudes anddistance values by processing the signal values of both signal paths insuch a way that the signal values are exchanged crosswise.
 5. The 3D TOFsensor according to claim 1, wherein optically sensitive surfaces of thepixels of the first channel (pixels A) and the pixels of the secondchannel (pixels B) are optically connected in such a way that bothpixels absorb the same amount of light.
 6. The 3D TOF sensor accordingto claim 3, wherein optically sensitive surfaces of the pixels of thefirst channel (pixels A) and of the pixels of the second channel (pixelsB) are connected by means of a diffuser that distributes an amount oflight evenly across optical surfaces of respectively one pixel fromchannel A (pixel A) and one pixel from channel B (pixel B).
 7. The 3DTOF sensor according to claim 1, further comprising a comparator stepdesigned to compare the signal values present in parallel at outputs ofthe pixel matrix with each other and/or with a default value ascomparator threshold value.
 8. The 3D TOF sensor according to claim 3,further comprising two analog-to-digital converters that are designed todigitize analog signal values A and B in the respective signal path A orsignal path B and that are respectively connected to the logic of signalpath A or the logic of signal path B.
 9. The 3D TOF sensor according toclaim 1, wherein the 3D TOF sensor is designed to test a functionalityof photo receivers by comparing the signal values A and the signalvalues B in an additional special signal processing step; it does so byactivating the pixels A and the pixels B at the same time thetransmitters are started.
 10. The 3D TOF sensor according to claim 1,wherein the 3D TOF sensor is designed to test a functionality of thetransmitters by a further additional signal step; and does so byactivating either the pixels A or the pixels B at the same time thetransmitters in only one transmitter group are started.
 11. The 3D TOFsensor according to claim 1, wherein each signal path comprises aprocessor which is connected to the signal path of the logic of thechannel, wherein calculated amplitudes and distance values, which weretransmitted by the respective logic, are transmitted to each processor.12. The 3D TOF sensor according to claim 11, wherein one of theprocessors is a safety-related processor that has at least twoindependent reactionless computing cores and processes two inputs on oneoutput (1oo2—one out of two).
 13. A 3D TOF sensor with an image sensorthat comprises an image recorder with a pixel matrix, wherein the imagerecorder can read out image data in individual, at least two, separateindependent blocks of the pixel matrix and transfer the image data in atleast two separate signal paths, the pixels of the pixel matrix of theimage recorder comprise at least one photosensitive receiver and amemory cell for storing charge carriers, and the signal paths have ananalog-to-digital converter, a programmable logic, memory cells and aprocessor connected to them, wherein the image recorder comprises atleast one global shutter for all blocks that can be released by acontrol signal of the programmable logic and can set a minimum shuttertime of less than 100 ns, and the signal paths are wired independentlyand reactionlessly from each other, and the signal paths are wired suchthat the data of the logics or the processors can be exchangedcrosswise.
 14. The 3D TOF sensor according to claim 13, wherein eachblock of the image recorder is assigned its own imaging optics in such away that the same image of an object side is displayed on the blocks,and that at least one light source is assigned to each block, which issynchronized with the global shutter via the logic.
 15. The 3D TOFsensor according to claim 13, wherein the image sensor features aseparate global shutter in each block that can be controlled by thelogic of assigned signal paths.